1. Field of the Invention
The present invention relates to a technology for processing an image captured by an optical system having a wide angle and a large distortion.
2. Description of the Related Art
A vehicle is provided with a fender mirror, a door mirror, and a back mirror so that blind spots that cannot be seen directly by a driver can be seen; however, it is difficult to see the blind spots perfectly over the entire circumference of the vehicle only by these mirrors. To view the blind spots that cannot be seen even with these mirrors, developments of an on-board visual camera fitted to the front, rear, or sides of the vehicle have been underway recently (see, for example, Kenichi HATANAKA, et al., “Development of In-Vehicle Image Processing Equipment” SEI technical review, vol. 162, pp. 90-93, March 2003).
In the case of a front camera, 180 degrees or more is required as a view angle of a camera to confirm a vehicle coming from left and right at the time of approaching an intersection. Also with a rear camera, an area that a driver wishes to view is a central image of the camera at the time of moving back. Even at this time, by displaying the left and right blind spots, the vehicle can be moved back more safely. For this purpose, 180 degrees or more is required as the view angle of the camera. The side camera fitted in the fender mirror or the door mirror, or to the vicinity of the side mirror is mainly for viewing blind spots of sides of the vehicle. To continuously view a close image of front wheels or rear wheels, and a forward or rearward image thereof with the side camera from immediately below the side, a camera having a view angle of 180 degrees or more is required as well. However, with the camera having the view angle of 180 degrees or more, an image of a central part becomes small, and there are such problems that recognition of a subject becomes slower and it is difficult to gain a sense of distance. Further, as the view angle becomes wider, the distortion becomes larger, thereby making it difficult to design an optical system with a small aberration.
Accordingly, as for a camera with the view angle being 140 degrees or more, distortion correction is generally performed for correcting a distortion of the image together with image processing. However, in the case of the view angle being 180 degrees or more, it is difficult to correct the distortion completely without any sense of strangeness.
Further, in the case of a camera using a sensor having an aspect ratio of 4:3, if a stereographic projection system uses equiangular projection or the like, a vertical view angle becomes approximately 140 degrees. If the front camera is installed at the right front, or the rear camera is installed toward the right back, right under the camera is out of a light receiving area of the sensor and becomes a blind spot. An image just under the camera, that is, an image showing a bumper coming close to an obstacle such as a wall at front or rear is one of images most desired to view when the vehicle is parked. If the camera is installed downward to see the image, the most part of a screen becomes an image showing a road, and a forward image as well as a downward image becomes small.
Therefore, image processing referred to as overview transformation (viewpoint transformation) for virtually moving a position of the camera according to calculation is performed. Conventionally, when distortion correction and overview transformation are to be performed, distortion correction is performed taking into consideration a distortion amount of the circumference of the lens, and thereafter, the viewpoint transformation is performed in order to conduct the overview transformation, thereby obtaining an overview image. In this system, due to continuous processing of distortion correction and viewpoint transformation, the processing becomes very heavy, and much calculation resources are consumed.
Correction of a magnification chromatic aberration and the distortion is performed by transforming a coordinate of each pixel in an original image by using a correction formula (a formula of coordinate transformation) such as a polynomial. Conventionally, various correction formulae have been known, and a conventional general formula for correcting the distortion can be expressed as
                    {                                                            X                =                                  x                  +                                      f                    ⁡                                          (                                              x                        ,                        y                                            )                                                                                                                                              Y                =                                  y                  +                                      g                    ⁡                                          (                                              x                        ,                        y                                            )                                                                                                                              (        1        )            where X and Y represent coordinates of a transformation source, x and y represent coordinates of a transformation destination, and f and g are correction functions.
As shown in Equation (1), a distortion-corrected image has been heretofore obtained by performing coordinate transformation vertically and horizontally symmetrically around the center of the image. In this case, the image generally has a sense of strangeness at four corners, and when there is a subject desired to see at an upper or lower corner, a natural correction image cannot be obtained.
Further, when the distortion correction and the overview transformation are to be performed, after the distortion correction is performed, the viewpoint transformation for the overview transformation is performed separately. Accordingly, the processing becomes very heavy, and much calculation resources are required.